Skip to content

WIP: Surface filter-promoted unraisable warnings directly (#14263)#14499

Draft
paulzuradzki wants to merge 7 commits into
pytest-dev:mainfrom
paulzuradzki:bugfix/14263-unraisable-warning-direct-raise
Draft

WIP: Surface filter-promoted unraisable warnings directly (#14263)#14499
paulzuradzki wants to merge 7 commits into
pytest-dev:mainfrom
paulzuradzki:bugfix/14263-unraisable-warning-direct-raise

Conversation

@paulzuradzki
Copy link
Copy Markdown

@paulzuradzki paulzuradzki commented May 19, 2026

Summary

Closes #14263.

filterwarnings = error::ResourceWarning does not fail tests that leak resources through a reference cycle. collect_unraisable wraps the captured warning in PytestUnraisableExceptionWarning, a class the user has not filtered.

collect_unraisable now checks warnings.filters before wrapping. When an active error filter targets a class that the captured warning is an instance of, collect_unraisable re-raises that warning directly; the resulting exception fails the test.

Visual

Behavior on main: this test passes despite user configuring for ResourceWarning to trigger an error. Pictured test_leak.py triggers ResourceWarning.

image

After PR: raises error as expected

image

Scope

collect_unraisable chooses between two branches when it processes a queued unraisable:

  • Unwrap (new in this PR): re-raise the unraisable's inner warning class directly, so the user's error::<class> filter fails the test as written. Applies only when an active error::<class> filter matches that class.
  • Wrap (pre-PR behavior, still the default): wrap the unraisable in PytestUnraisableExceptionWarning and emit it via warnings.warn. A __del__ that raises a Warning without a matching error::<class> filter still hits this branch.

Suites without error::<warning> filters see no change.

The unwrap-branch filter check is action == "error" plus issubclass(warning_class, filter_category). It does not re-run Python's warnings module's full match logic, which additionally tests the filter's message_regex, module_regex, and lineno fields. Consequence: for users with narrowly-scoped filters like error:some_msg:ResourceWarning::42, this branch can fail a test on a ResourceWarning whose message wouldn't have matched the regex. For broad filters like error::ResourceWarning the behavior matches user intent.

Branch structure

Three red→green commit pairs plus a comment-cleanup commit. Each red commit fails the named test; the next commit makes it pass.

  1. 92c7681a2 (red) → 4cf46d4f9 (fix): test_refcycle_resource_warning_filter exercises the user-visible wrap-vs-unwrap behavior covered in Summary.
  2. 21d334ad1 (red) → 839a20a0c (fix): the move-GC hardening from Fix session-end gc in unraisableexception plugin to respect warning filters #14273. test_unraisable_decouples_from_cleanup_stack_order uses @hookimpl(trylast=True) to force a LIFO cleanup-stack order that PR apply warnings filter as soon as possible, and remove it as late as possible #13057 incidentally avoided. The fix moves gc.collect() and queue processing into pytest_unconfigure, which runs before _cleanup_stack.close().
  3. dead24a90 (red) → ef0d16180 (fix): test_pytest_unconfigure_survives_failed_pytest_configure raises UsageError from a conftest's pytest_configure, which exposed a KeyError after step 2 moved the GC call out of config.add_cleanup. The fix adds a stash-presence check at the top of pytest_unconfigure.

Cleanup: 56f71b785 tightens the new comments in unraisableexception.py.

Relationship to #14273

#14273 attempted approach 2 from the issue (move GC to pytest_unconfigure). Maintainers closed it unmerged: #13057 (Dec 2024) had already shipped approach 1, so the move-GC change alone produced no observable behavior difference and no fails-then-passes regression test was possible. @Zac-HD asked: "we'd need to see a regression test which demonstrates that the unraisable hook is now subject to warnings."

Step 2 in the branch structure addresses that question. The red test forces the LIFO ordering that #13057 incidentally avoided, demonstrating the structural fragility was load-bearing on plugin registration order rather than benign.

Testing

Reproducer

My adapted version of the original gist from the issue. The original references an undefined stderr_lines in its run_scenario helper; the adapted version applies a one-line fix.

For a one-file check, save this as test_leak.py:

import gc; gc.disable()  # let the cycle survive to session-end gc

def test_it():
    f = open(__file__)
    cycle = [f]
    cycle.append(cycle)

Then run pytest -W error::ResourceWarning test_leak.py. The -W flag drives the same code path as the pytest.ini filterwarnings setting. On main the test passes (bug); on this branch it exits 1 with the inner ResourceWarning raised directly, no PytestUnraisableExceptionWarning wrapper.

If you run from inside the pytest source tree, the repo's pyproject.toml [tool.pytest] filterwarnings = ['error', ...] promotes every warning to an error. Comment out the bare 'error' entry, or run the snippet from outside the repo.

  • Regression tests
  • Manually verify bug on main
    • Check out main, install (pip install -e .), run the adapted reproducer. Scenario 1 exits 0 (bug confirmed). Scenario 2 exits 1.
  • Manually verify branch fix. Check out branch, run the same reproducer. Scenario 1 now exits 1 with the inner ResourceWarning raised. Scenario 2 still exits 1.
  • Check out any of the red commits (92c7681a2, 21d334ad1, dead24a90); the named test fails. Check out the next commit; it passes.

External suite tests

(time-permitting; I don't think this needs to block review)

  • Verify the scope guard. Run a suite that raises a Warning from __del__ without a matching error filter against this branch.
  • Run a real project that uses error::DeprecationWarning or error::UserWarning filters against this branch. Nothing previously-passing should now fail.

AI disclosure

Side note: open to feedback to pull some of the PR comments into the code if helpful

Changelog

changelog/14263.bugfix.rst.

@psf-chronographer psf-chronographer Bot added the bot:chronographer:provided (automation) changelog entry is part of PR label May 19, 2026
@paulzuradzki paulzuradzki force-pushed the bugfix/14263-unraisable-warning-direct-raise branch 2 times, most recently from d7db869 to 476e4f5 Compare May 19, 2026 21:20
filterwarnings = error::ResourceWarning does not fail tests that leak
resources through a reference cycle. collect_unraisable wraps the
captured ResourceWarning in PytestUnraisableExceptionWarning, a class
the user has no filter for, so the run exits 0.

This test pins that contract: on a refcycle-leaking test with
error::ResourceWarning configured, pytest should exit non-zero and the
output should show the inner ResourceWarning rather than the wrapping
PytestUnraisableExceptionWarning. Fails at this commit; the next
commit ships the fix that turns it green.

Refs pytest-dev#14263.
When sys.unraisablehook captures a Warning subclass instance and the
user has an active ``error::<that class>`` filter, raise the original
warning rather than wrapping in PytestUnraisableExceptionWarning. The
wrap path remains for any case where no matching error filter is set,
so suites that don't use ``error::<warning>`` filters see no change.

Filter matching is approximate: category only, not message/module/lineno.
The check errs toward false negatives, never false positives.

The regression test added in the previous commit now passes. Additional
coverage:

- test_refcycle_userwarning_filter: locks the contract for a non-builtin
  Warning subclass.
- test_unraisable_warning_without_filter_still_wraps: scope guard. A
  Warning raised from __del__ without a matching error filter must
  still be wrapped, not raised directly.
- test_unraisable_warning_filter_add_note_dedups: covers the duplicate-
  note guard in the unwrap path for singleton/cached Warning instances.

Tightens the ``errors`` list type from list[Exception] to
list[Warning | RuntimeError]. Adds Paul Zuradzki to AUTHORS. Notes in
test_create_task_raises_unraisable_warning_filter that the propagated
class is now bare RuntimeWarning rather than the wrapping
PytestUnraisableExceptionWarning (because -Werror activates the new
unwrap path).

Closes pytest-dev#14263.
@paulzuradzki paulzuradzki force-pushed the bugfix/14263-unraisable-warning-direct-raise branch from 476e4f5 to 4cf46d4 Compare May 19, 2026 21:39
Copy link
Copy Markdown

@FuzzysTodd FuzzysTodd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pytest_unconfigure fires before _cleanup_stack.close(), so warning
filters managed via the cleanup stack (the warnings plugin's
catch_warnings context, in particular) are guaranteed active when GC

@paulzuradzki paulzuradzki changed the title Surface filter-promoted unraisable warnings directly (#14263) WIP: Surface filter-promoted unraisable warnings directly (#14263) May 20, 2026
The previous arrangement works because PR pytest-dev#13057 (Dec 2024) reordered
default_plugins so the unraisableexception cleanup pops before the
warnings plugin's catch_warnings exit under LIFO. Correctness then
relies on registration order; any plugin that registers a cleanup
later than unraisableexception's and resets warnings.filters revives
the original bug.

The test uses a conftest with @hookimpl(trylast=True) to register a
warnings.resetwarnings cleanup after the built-in plugins. Under LIFO,
that cleanup pops first; warnings.filters is empty by the time
unraisableexception runs gc.collect() and the leaking finalizer fires.
warnings.warn then emits the ResourceWarning silently, since no filter
promotes it, and the suite exits 0.

Fails at this commit. The next commit moves gc.collect() and
unraisable processing into pytest_unconfigure, which runs before
_cleanup_stack.close(); the test then passes regardless of plugin
registration order.

Refs pytest-dev#14263.
Register only the hook-restore + stash-cleanup as the
config.add_cleanup callback. Move the GC pump and collect_unraisable
call into a new pytest_unconfigure(config) hook.

pytest_unconfigure fires before _cleanup_stack.close(), so warning
filters managed via the cleanup stack (the warnings plugin's
catch_warnings context, in particular) are guaranteed active when GC
runs. This decouples the unraisable step from plugin registration
order in default_plugins. The previous arrangement worked only because
of LIFO ordering on _cleanup_stack; pytest-dev#13057 (Dec 2024) reordered
default_plugins to make that ordering correct, but the structural
fragility remained.

No observable behavior change. The 141 existing tests in
test_unraisableexception + test_warnings + test_recwarn +
test_threadexception still pass. The previous commit's
test_refcycle_resource_warning_filter continues to fail on main and
pass here.

This is the structural side of the issue's two proposed fixes; the
user-visible side shipped in the previous commit.
The previous commit moved gc.collect() and unraisable processing from
the config.add_cleanup callback into pytest_unconfigure. pytest calls
pytest_unconfigure for every plugin during shutdown, regardless of
whether that plugin's pytest_configure completed. When another
plugin's pytest_configure raises pytest.UsageError, pluggy stops
calling the remaining configure hooks; unraisableexception's
pytest_configure never runs and its stash key stays unset.

Without a presence check at the top of pytest_unconfigure, the
unraisable plugin subscripts config.stash[unraisable_exceptions] on
an unset key, hits KeyError, and pytest reports INTERNALERROR
instead of USAGE_ERROR. The user sees a pytest-internal traceback
instead of the plain "ERROR: <their bad config>" message.

The test installs a conftest whose pytest_configure raises
UsageError, then asserts the run exits with USAGE_ERROR and that
no INTERNALERROR or KeyError appears in stderr. Fails at this
commit; the next commit adds the guard.

Refs pytest-dev#14263.
When another plugin's pytest_configure raises (e.g. pytest.UsageError
in testing/acceptance_test.py::test_config_error), pluggy skips
remaining configure hooks. unraisableexception.pytest_configure never
runs, config.stash[unraisable_exceptions] is never set. The previous
config.add_cleanup callback wasn't registered in that case either, so
cleanup was a no-op. The pytest_unconfigure hook introduced in the
previous commit ran unconditionally and hit KeyError on the unset
stash, surfacing as INTERNAL_ERROR where pytest should exit with
USAGE_ERROR.

Guard with a stash-presence check at the top of pytest_unconfigure.
test_config_error catches the regression direction.
Drop "drain" in favor of literal language. The pytest_unconfigure
comment now states what the hook depends on (filters managed via the
cleanup stack, garbage-collected finalizers) rather than naming the
previous mechanism. The guard comment says "the queue stash was
never set" instead of the metaphorical "nothing to drain."
@paulzuradzki paulzuradzki force-pushed the bugfix/14263-unraisable-warning-direct-raise branch from 63a5109 to 56f71b7 Compare May 24, 2026 12:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bot:chronographer:provided (automation) changelog entry is part of PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Session-end gc.collect() in unraisableexception plugin runs after warning filters are torn down, silently losing ResourceWarnings

2 participants